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EPRRI, funded by the US. Department of 
Education’s Office of Special Education Programs, 
investigates the impact of new educational accounta- 
bility systems on students with disabilities and on 
special education. EPRRI addresses the research needs 
of policy-makers and other key stakeholders by identi- 
fying critical gaps in current knowledge, seeking prom- 
ising strategies, and publishing Topical Reviews, Policy 
Updates, and Issue Briefs. The Institute is a joint 
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Topical Review Highlights 



Public reporting of state assessment results - for all stu- 
dents - is a fundamental element of accountability and edu- 
cation reform. Stakeholders ( including school staff, school 
boards, parents, policymakers, and the media) need accessi- 
ble data in order to improve instruction and document bow 
well students are doing. Reporting of results for students 
with disabilities is particularly important because it helps 
ensure that these students are included in accountability sys- 
tems. 

Both the Individuals with Disabilities Education Act 
(IDEA), and Title 1 of the Elementary and Secondary 
Education Act mandate reporting on the performance of stu- 
dents with disabilities. However, the form and content of 
reports vary widely among states and also within states, 
where reports are also published at the school and district 
levels. 

According to the National Center on Educational 
Outcomes (NCEO’s) ongoing analysis of 1999-2000 state 
assessment reports for students with disabilities: 

• States are moving toward, but have not yet fully com- 
plied with the IDEA directive to report on the participation 
and performance of students with disabilities with the 
same frequency and in the same detail as for other stu- 
dents; and 

• Only one state, Kentucky, has so far included any infor- 
mation on alternate assessments for students with disabil- 
ities. 

To illustrate the variations in assessment reports, this 
Topical Review examines in detail the reporting practices of 
Texas, New York, Maryland, and California. These states dif- 
fer in assessments, standards, the grades at which students 
are tested, and the ways test results are used. 

This review also identifies several important issues in 
the reporting of assessment data for students with disabili- 
ties: 



• Assessment results for many students are still not 
reported, including data for some students using non- 
approved accommodations, using alternate assessments, 
or taking tests designed for lower grade levels. The states 
that do report these data often report them separately from 
scores of students without disabilities. 

• Alternate assessments are new, and states must face the 
technical challenge of aggregating these results with regu- 
lar scores in a way that is statistically sound States must 

also decide on the purpose and focus of alternate assessments. 

• As states track performance results over time, they need 
to document and account for changes in students’ special 
education status. States need to find ways to report on the 
performance of all students with disabilities, while also 
following the longitudinal performance of clearly defined 
target groups of these students. 

In 1997, NCEO developed a framework for states to use 
when developing reporting policies and practices. The guid- 
ing principles included providing data from all test takers, 
giving the rates of (as well as the reasons for) the exclusion 
of students with disabilities, keeping records of accommoda- 
tions used, and informing parents about the reporting policy 
for their child’s data. NCEO’s newly updated framework calls 
for reporting the scores of all students with disabilities, 
whether they participate with or without accommodations or 
use an alternate assessment. Students whose scores are not 
aggregated with others, or who are not in the assessment sys- 
tem in any way should still be accounted for. 

The reports to stakeholders must be readily available, 
clear, comprehensive, and concise. They should provide 
comparative information about schools, districts, states, 
regions, or standards, including changes over time. In addi- 
tion, they should maintain confidentiality and offer cautions 
against misinterpretation. 

Reporting assessment performance data is a critical step 
in education reform. The next important step is to use the 
information to improve outcomes and achievement levels for 
all students. 
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1 . Introduction 



Public reporting of state assessment results is the most 
basic form of accountability. It has become an important tool for 
holding public schools accountable for student attainment of 
higher educational standards, hi the past, educators often made 
decisions based on “gut instincts” about how students were 
doing. In today’s complex world, instincts no longer are a reli- 
able method of decision-making. Educators and other stake- 
holders want to use data as a basis for making sound decisions 
about student achievement and for holding schools accountable. 
Technology has enabled schools to collect huge amounts of data 
that need to be organized into a format that is accessible and 
understandable (Kongshem, 1999). 

Public reporting on the assessment performance of stu- 
dents serves several purposes (Cibulka & Derlin, 1995). One 
major purpose is to improve instruction. Both educators and the 
public want to verify, with solid data, their instincts about what 
works in education. Data need to be reported in accessible for- 
mats so that all stakeholders can gain valuable knowledge about 
best practices that boost student achievement and school effec- 
tiveness. Another major purpose of public reporting is to docu- 
ment how well students are doing. An effective reporting system 
must provide assessment performance information to both edu- 
cators and the public. Further, public reporting is a way to share 
information for accountability purposes. Brown (1999) also 
lists the purposes of public reporting, but they are slightly differ- 
ent informing the public; allowing district leadership to monitor 
progress toward goals; and giving individual schools a chance to 
highlight accomplishments. 

One of the most difficult reporting decisions is “what” to 
report. Keman-Schloss (1999) says that most states and districts 
“typically report what they think is important for the public to 
know” (p. 46). He suggests that rather than using a “top-down, 
here’s-what-we’re-going-to-tell-you” approach, stakeholders 
need to be asked some simple questions like, “What counts? 

What do you need to see to be persuaded that schools are 
improving? What measures should we use to measure progress 
and hold schools accountable?” (p. 46). 



Almost all states now publish at least one statewide educa- 
tional accountability report on the condition of public education 
(Bielinski, Thurlow, & Callender, 2001). Some states have five 
or six reports. The reports vary widely in their format and the 
types of information they contain. Some feature achievement test 
results for schools or districts, in addition to overall state-level 
results. Many states also include non-test performance meas- 
ures, such as attendance (39 states), dropout rates (37 states), 
graduation rates (27 states), and enrollment (38 states) 

(Goertz, Duffy, & LeFloch, 2001). Other states expand their 
reporting to include measures of educational inputs and 
processes, such as per-pupil expenditures, student-teacher 
ratios, and other indicators believed to have a direct impact on 
student outcomes (Barber, Paris, Evans, & Gadsden, 1992; 
Erickson, Ysseldyke, Thurlow, & Elliott, 1997). According to 
Cibulka and Derlin (1995): 

Wide variation can be found in the stated purpose of 
reporting, the information included, the unit of analysis 
(state, districts, or schools), comparisons made (against 
state verages, against peers, against past performance), 
information distributed, and the uses to which it is put by 
state and local policymakers (p. If). 

The Study Group on Education Accountability, organized by 
the National Association of State Boards of Education (NASBE, 
1998), developed a framework of ten standards for guiding dis- 
cussion, design, and evaluation of state and local education 
accountability systems. The fifth standard refers specifically to 
reporting assessment performance information: “Those respon- 
sible for governing accountability regularly report student and 
school performance information in useful terms and on a timely 
basis to school staff, students and their families, state and local 
policymakers, and the news media” (p. 7). NASBE identified six 
indicators that demonstrate progress toward this standard: 

1. School wide scores for student performance are analyzed 
in several ways, including absolute performance in relation 
to standards, degree of improvement over previous per- 
formance, and in comparison with predicted scores based 
on contextual conditions. 
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2. Student performance data are reported in aggregated 
school-wide averages and in disaggregated form to draw 
attention to defined subpopulations whose performance 
merits particular attention, such as students with disabili- 
ties, students eligible for free or reduced price lunch, stu- 
dents of different radal/ethnic backgrounds, and students 
with special language needs. 

3- Student scores are provided to teachers for their individual 
students within the same academic year in which the stu- 
dent takes the assessment and are used to improve classroom 
practice. 

4. Performance reports are produced using language and for- 
mat appropriate to various intended audiences. Reports 
include thorough explanations of the meaning of the 
results, the limitations of the data, and important student 
and school contextual factors that may affect student 
achievement 

5. Education decision makers use student performance 
results as guides for improving policy and practice. 

6. Operation of the accountability system maintain s the confi- 
dentiality of individual students. 



Ultimately, according to the Education Commission of the 
States (1998), an accountability system has four elements: 

1 .Measurement of student success at state standards; 

2. Reporting to inform districts, schools, parents and neigh- 
borhoods; 

3. Labeling to designate schools for rewards, assistance, or 
dramatic remedial action; and 

4. Remedies to help children in low-performing schools by 
supporting teachers and principals who know how to 
improve, providing guidance and help when schools floun- 
der, and replacing schools that cannot improve. The 
accountability system also includes review of assistance 
measures to be sure they are working (p. 3). 

The purpose of this paper is to provide a comprehensive 
account of reporting the performance of students with disabili- 
ties on state assessments. Following an overview of issues, this 
paper provides an explanation of the laws that specifically gov- 
ern the reporting of assessment results (Individuals with 
Disabilities Education Act Amendments of 1997 and Title I of the 
Elementary and Secondary Education Act of 1994). Next it 
explores die levels and audiences of reports that are traditional- 
ly used in education. Current reporting practices across all 
states are described generally, and the practices in four states 
are examined in greater depth. After discussing several issues 
related to reporting assessment performance, the paper con- 
cludes with an exploration of frameworks and implications for 
reporting on the state assessment performance of students with 
disabilities. 
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Reporting on the State Assessment Performance of Students with Disabilities 



2. Overview of Reporting on 
State Assessment Performance 



Reporting information on the state assessment performance 
of students with disabilities is important because it ensures that 
they are included in the accountability system. Failure to report 
assessment results is one of the most common ways in which 
students with disabilities have been excluded from educational 
accountability (Erickson et al., 1997). Exclusion from reports 
sends the message that these students are not important - that 
they do not count, which is an inappropriate message at a time 
when it is extremely important to document the performance of 
all students and to emphasize accountability for learning. The 
way in which data are reported can affect the way students are 
perceived. Students with disabilities are part of the total student 
body and should be treated as such. In the same way, their data 
must be treated as part of the data from the total student body. 
Performance reports provide concrete evidence of student 
achievement, so it is vital that reports include information that 
demonstrates that all students, including those with disabilities, 
are receiving a standards-based, rigorous academic education. 

Accountability for the educational performance of students 
with disabilities who receive special education services is evolv- 
ing from compliance with input and process requirements to 
responsibility for student outcomes. One of the primary ways 
that students with disabilities have been included in general edu- 
cation accountability systems is through the inclusion of their 
test scores in state and district assessment performance reports 
(Roach, Daily, & Goertz, 1997). Although education statistics 
have been reported in the United States since the 19th century 
(with reports on national and state-level education indicators 
reported since the mid-1980s) (Blank, 1993), few states pub- 
licly reported the educational assessment performance results of 
students with disabilities until recently. In fact, most state agen- 
cies did not even keep track of the rate at which these students 
participated in testing or were included in any state or local 
accountability indices. Low rates of assessment participation and 
variability from one place to the next prevented policy-relevant 
conclusions from being drawn about the extent to which stu- 
dents with disabilities benefited from their educational experi- 
ences (Erickson et al., 1997; Roach & Raber, 1997). 

States have faced both technical and political challenges as 
they work toward the inclusion of students with disabilities in 
state assessment and accountability systems (Goertz et al., 

O 



2001). These challenges underlie decisions policymakers must 
make about who gets tested, whose test scores are reported, 
and whose scores are included in accountability measures. One 
of the greatest technical challenges is reporting accurate and 
reliable information on student mastery of state standards under 
nonstandard assessment conditions, often produced by the use 
of some accommodations and participation in alternate assess- 
ments (Goertz et al., 2001). 

There are several issues involved in reporting the perform- 
ance of students with disabilities on state assessments. A focus 
group held in Texas identified these seven issues (Texas 
Education Agency, 1995): 

• There are concerns about students whose assessment 
results are reported but not included in the accountability 
system. 

• Some believe that the data for students with disabilities 
should be reported separately from the data for other stu- 
dents. 

• Categorical reporting of results by disability is desired by 
some but not others, and is seen as raising issues of confi- 
dentiality. 

• There are concerns related to reporting results of students 
who are not attending their home campus. 

• There are questions about whether results, particularly 
regarding students with disabilities, are communicated in a 
way that is clear and easily understood. There is disagree- 
ment about whether data obtained when students use 
accommodations or alternate assessments should be 
reported separately from data obtained under standard 
assessment procedures. 

• There is a question about the kinds of scores (e.g., absolute 
scores or change in performance over time) that should be 
reported for students with disabilities. 
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Literature on Assessment Performance Reporting 

Information on assessment performance reporting is 
almost always buried within literature focused on other topics. 
In searching the literature, we found that simply using the 
search word “reporting” produced very few documents. Thus, 
we searched more broadly - in the areas of “accountability” and 
“school improvement” - where we found a large body of litera- 
ture that included sections on assessment reporting. 

A massive volume of literature exists on the need for 
schools to use assessment performance information in the deci- 
sion making process. Much of this literature Ms under the gen- 
eral category of school improvement. For example, an ERIC 
data-base search, conducted in September 2001 using the term 
“school improvement,” retrieved 32,238 citations. An advanced 
search using both the terms “school improvement” and “per- 
formance reporting” resulted in 1,905 citations. School 
improvement literature often discusses how schools can use 
data-driven leadership to increase and hold schools accountable 
for student achievement It also frequently mentions how various 
stakeholders can use performance reports, but seldom provides 
much information about what should be included in the reports. 



A body of literature also exists that discusses the content of 
performance reports. Since the final authority for educational 
policy lies at the state level, much of this literature is published 
by state agencies. These publications explain what types of per- 
formance data are required to be reported by state law 
(Holcomb, 1999). Associations within a state then often publish 
materials to help their members comply with the law. For exam- 
ple, the Minnesota School Board Association published School 
District Accountability: Using Data to Make Decisions 
(2000). Since there are 50 states, a huge volume of state-specif- 
ic technical assistance literature has been published. 

There is a dearth of research-based literature, however, that 
is national in scope and that examines the ways in which infor- 
mation can be contained in performance reports for all stu- 
dents, regardless of whether they have disabilities. Even though 
states are required to report the performance of students with 
disabilities on state assessments, there is very little published 
about how performance reports should include information 
about these students. Thus, though federal laws require state 
assessment results of students with disabilities to be disaggregat- 
ed and publicly reported, several states continue to struggle in 
their efforts to do so. 
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3. Laws Governing Reporting of 
Assessment Results for Students with Disabilities 



Public reporting of state assessment results is becoming an 
increasingly important tool for ensuring that public schools are 
accountable for helping students meet higher educational stan- 
dards. There are two primary federal mandates that emphasize 
the importance of establishing on-going assessment reporting 
systems that include all students. These are the 1997 reautho- 
rization of the Individuals with Disabilities Education Act, which 
provides for special education and related services, and the 
Elementary and Secondary Education Act , which provides for 
Title I and other programs. 

Individuals with Disabilities Education Act (IDEA) 

In 1997, the IDEA was reauthorized, with amendments 
directed at reporting the performance of students with disabili- 
ties in state and district assessments. The new assessment 
reporting requirements are reflected in two parts of the amend- 
ed law: (1) in the specific requirements for participation in gen- 
eral assessments and in newly developed alternate assessments, 
and (2) in the requirements for performance goals and indica- 
tors. 

Reporting requirements: The state educational agency 
makes available to the public, and reports to the public with 
the same frequency and in the same detail as it reports on the 
assessment of nondisabled children, the following: 

(1) The number of children with disabilities participating in 
regular assessments. 

(2) The number of those children participating in alternate 
assessments. 

(3) (I) The performance of those children on regular assess- 
ments (beginning not later than July 1, 1998) and on 
alternate assessments (not later than July 1, 2000), if 
doing so would be statistically sound and would not 
result in the disclosure of performance results identifi- 
able to individual children. 

(n) Data relating to the performance of children 
described under subclause I shall be disaggregated 
(aa)for assessments conducted after 
July 1, 1998; and 

(bb)for assessments conducted before July 1, 1998, if 
the State is required to disaggregate such data prior to 




Performance goals and indicators: States must define 
performance goals and indicators for themselves, and then pres- 
ent data for these to the Secretary of Education and the public 
every two years. Then, based on an assessment of progress, the 
state may need to revise its’ State Improvement Plan. One of the 
required indicators is the performance of students with disabili- 
ties on state and district assessments. Others are high school 
graduation rates and dropout rates (612) (a) ( 16 ). 

Title I of the Elementary and Secondary Education Act 
of 1994 

This law was amended in 1994 in response to concerns 
about the impact of Title I over the previous 25 years: low 
expectations for educationally disadvantaged students, an 
instructional emphasis on basic skills, isolation from the regular 
curriculum, and a focus on procedural compliance rather than 
academic outcomes (Goertz et al., 2001; U.S. Department of 
Education, 1993). Among its other reporting requirements, Title 
I now includes the requirement that programs report student 
performance on the state assessment, and that the performance 
of students with disabilities be disaggregated. Jaeger and Ticker 
(1998) state: 

When the United States Congress passed the Improving 
America’s Schools Act of 1994, it created an obligation and 
an opportunity for states, local educational agencies, and 
schools to disaggregate and analyze data resulting from 
assessments of their students’ achievement in ways that 
would be useful and illuminating, (p. 1). 

According to Title I, in the 2000-01 school year, “each State 
must have in place a Statewide assessment system that serves as 
the primary means for determining whether schools and dis- 
tricts receiving Title I funds are making adequate yearly 
progress toward educating all students to high standards” 
(Summary Guidance on the Inclusion Requirements for Title I 
Final Assessments, April 4, 2000) .Title I requires school district 
or school-level performance reports to contain the scores of 
students with disabilities for purposes of public reporting and 
school and district accountability. Table 1 contains the Title I 
assessment reporting requirements. 
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Table 1. Tide I Assessment Reporting Requirements 
Elementary and Secondary Education Act 
of 1994, P.L 103-382 (1994) 

Each state plan shall demonstrate that the State has 
developed or adopted a set of high quality, yearly stu- 
dent assessments in at least mathematics and reading 
or language arts, that will be used as the primary 
means of determining the yearly performance of each 
local educational agency and school served under this 
part.. .Such assessments sbaU...enable results to be dis- 
aggregated within each State, local educational agency, 
and school by gender, by each major racial and ethnic 
group, by English proficiency status, by migrant status, 
by students with disabilities as compared to nondis- 
abled students, and by economically disadvantaged 
students as compared to students who are not eco- 
nomically disadvantaged 
[Sec. 1111 (b) (3) (I) (A)] 

Each local educational agency receiving funds under 
this part shall — ...publicize and disseminate to teach- 
ers and other staff, parents, students, and the commu- 
nity, the results of the annual review under paragraph 
(2) of all schools served under this part in individual 
school performance profiles that include statistically 
sound disaggregated results as required by section 
1111 (b) (3) (1) [which specifies students with disabil- 
ities]; and provide the results of the local annual 
review to schools so that the schools can continually 
refine the program of instruction to help aU children 
served under this part in those schools meet the State’s 
student performance standards. 

[Sec. 1116 (a) (3) (4)] 



The explicit purpose of the ESEA is to support the broad- 
based reforms occurring in states and localities and to extend 
high educational expectations to students served by Tide I and 
other ESEA programs (Guidance on Standards, Assessments and 
Accountability, 2000). Rather than a separate Title I system, the 
standards and assessments employed for Title I are the same as 
those developed by the state and local districts for all children. 
Because adequate yearly progress is based primarily on the 
state’s final assessments, the universe of students includes those 
in the grades assessed. 

The state is responsible for determining the methods and 
procedures for measuring adequate yearly progress. Adequate 
yearly progress as defined by a state describes the amount of 
yearly improvement each Title I school and district is expected 
to make in order to enable low-achieving children to meet high 
performance levels expected of all children. Each state’s defini- 
tion of adequate progress must be based primarily on its final 
assessment system included in the state’s plan. The concept of 
adequate yearly progress under the 1994 reauthorization of Title 
I includes (1) an emphasis on accountability of schools and 
LEAs receiving Title I funds (i.e., whether they are making ade- 
quate progress toward enabling their children to meet the state’s 
standards) rather than emphasizing the Title I program itself or 
even the yearly performance gains of participating children; and 
(2) a definition that holds LEAs and schools accountable for the 
amount of improvement they make each year (Guidance on 
Standards, Assessments and Accountability, 2000). 
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The Educational Policy 
Reform Research Institute 



According to a 50-state survey of state assessment and 
accountability systems conducted by the Consortium for Policy 
Research in Education (CPRE) between February and June 
2000, 39 of the 48 states with statewide assessment systems dis- 
aggregate test data by race/ethnicity and gender, with 35 states 
disaggregating data by socio-economic status (Goertz et al., 
2001). The CPRE study found that states disaggregate and report 
scores of students with disabilities in one of five ways: 

1. The states neither disaggregate nor report these scores; 

2. The states disaggregate but do not publicly report the 
scores; 

3. The states do not disaggregate but include the scores in 
aggregate score reports; 

4. The states report the scores of tests taken under standard 
conditions or under conditions that do not interfere with 
the comparability of -scores of students tested under regular 
conditions; or 

5. The states disaggregate and report all scores. These results 
may or may not be reported to the public. 

A detailed analysis of the ways in which students with dis- 
abilities are reflected in reports has been conducted by the 
National Center on Educational Outcomes (NCEO); the results of 
this analysis are presented later. 
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4. Levels of Reports 



Many types of performance reports are published, often 
created at different levels of the public education system. States, 
school districts, and schools all create public reports. Currently, 
all but 13 states produce annual school “report cards" or 
school profiles (Center for Community Change, 2001). These 
report cards provide their audiences with a range of data on 
schools. Some districts add local data to state reports (Cibulka 
& Derlin, 1995). States that take this approach are more likely 
to incorporate the data into their on-going planning and deci- 
sion processes. 

Most report cards are published on the state education 
agency website, and many states require them to be sent home 
to parents or printed in local newspapers (Center for 
Community Change, 2001). Some states continue to exclude stu- 
dents with disabilities from their accountability reports, while 
others exclude students who take tests with accommodations, or 
students who have used specific types of accommodations 
(Bielinsia et al., 2001). Among the more common levels at 
which public reports are produced are: (1) individual student 
reports, (2) school reports, (3) district reports, and (4) state 
reports. Each of these is addressed here. 

Individual Student Level Reports 

Performance reports at the individual student level are typi- 
cally used by districts and states to certify students for promo- 
tion or graduation, to inform families of their child’s progress, 
to guide student achievement, and to assist with school and 
classroom improvement (NASBE, 1998). Performance reports 
containing information on individual students are extremely use- 
ful for teachers, administrators, guidance counselors, school 
psychologists, and parents in diagnosing areas of relative 
strengths and weaknesses. The information can be used to 
develop instructional strategies that will benefit all students. 
However, as stated by Jaeger and Thcker (1998), “the test 
scores of individual students are the business of the student, the 
student’s parents, and the student’s teacher, and should not be 
reported in anyway that identifies the student” (p. 22). For 
obvious ethical and legal reasons, dam on individual students is 
.confidential and private, so the information contained in individ- 
ual student reports must be aggregated before being presented 
to a larger audience. A generally accepted rule is to refrain from 
reporting statistics for small groups of students (Jaeger & Thcker). 
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School Level Reports 

Most districts and some states use the school as the pri- 
mary reporting unit (NASBE, 1998). According to Education 
Week’s report “Quality Counts,” 36 states produced school-level 
report cards in 1999; this number has grown to 45 states in 
2001 (Quality Counts, 2001). The Center for Community Change 
(2001) identified a number of specific indicators that should be 
required on individual school report cards in all states: 

• Assessment scores, fully disaggregated as required by cur- 
rent Title I law, 

• Information about the quality of a school’s teaching staff, as 
measured (at least) by average years of experience, levels 
of degree attainment, numbers of inadequately licensed 
teachers, and measures of out-of-field teaching; 

• Average class size by grade; 

• Four-year graduation rates; 

• Disaggregated information on student suspensions and 
expulsions; 

• Indicators of overcrowding; and 

• Notification of whether the school has been identified as 
“low-performing.” 

Brickell and Paul (1999) argue that performance reports 
need to be accompanied by information about how performance 
has changed over time, how the performance compares with 
that of other similar organizations, and how the performance 
stacks up against absolute standards. Performance reports 
should enable stakeholders to measure student performance 
against benchmarked targets. Comparisons can be made to 
show progress toward standards at the school level. Sometimes 
these results are used in decisions about school accreditation 
(NASBE, 1998). 
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The assessment performance of all students must be report- 
ed because the success of all students is a vital component of a 
standards based accountability system. In order to ascertain 
whether assessment performance of all students is reported, it is 
important to know the percentage of students who participated 
in the assessment Assessment rates are calculated by dividing 
the number of students assessed by the number of students 
enrolled at the grade level tested. 

Individual schools provide assessment performance infor- 
mation, but it may not pertain to all of the individual students 
who are enrolled. For example, the presence of a special read- 
ing program at a school does not mean that every student in the 
school benefits from it Many of the students may have no con- 
tact at all with the special program. Thus, an analysis based on 
achievement of all the students in schools with the reading pro- 
gram may not reflect its actual impact on students, because par- 
ticipating students will be grouped together with students on 
which the program can have no direct effect. Similarly, a pre- 
dominance of teachers with few years of experience and a low 
mean achievement test score does not necessarily point to a 
cause and effect relationship between the two factors. It is 
unclear from the aggregate data whether the low achievers 
whose scores bring the mean down are actually in classrooms 
with inexperienced teachers (Hanson, Gardner, & McNamara, 
1986, p. 67). According to the Education Commission of the 
States (1998): 

Information about schools must be simple and easy to 
understand. School profiles should include student profi- 
ciency in reading, math and other subjects, as well as 
comparison with average scores of similar schools. 

Absolute score levels, however, should not be obscured by 
complex weighting schemes (p. 2). 

District Level Reports 

Performance reports at the district level may be used by 
some states in the same ways that school level reports are used 
by other states. The disadvantage of district level reporting is 
that schools within a district may vary a great deal in perform- 
ance, and most educational programs are delivered at the 
school, rather than at the district level (NASBE, 1998). 
Performance at the district level is often used to compare dis- 
tricts. For example: 



The mean score for all fifth graders in one district can 
easily be compared with the mean score for all fifth 
graders in another district and the superintendent will be 
bailed with either bouquets or brickbats depending on the 
standing of bis or her district (Hanson et al., 1986, p. 64). 

Often, however, districts hailed with “brickbats" claim that 
they were at an unfair disadvantage - too many economically 
disadvantaged, at-risk, limited English proficient students, or 
students with disabilities and inadequate resources (Cibulka & 
Deriin, 1995). Grouping districts with similar characteristics 
has been criticized because of lowered expectations or justifica- 
tion for existing performance differences. 

State Level Reports 

Studies on state level reporting at the National Center on 
Education Outcomes have found tremendous variability between 
states (Krentz, Thurlow, & Callender, 2000). Some states pro- 
duce five or six 500-page volumes annually, while others pro- 
duce a two- to three-page report A few states report state level 
data; others give school, district, and state level data annually. 
Accountability documents also vary in their focus. The indicators 
used in accountability reporting cover a wide spectrum, from 
detailed financial information to student mobility rates, and 
from staffing information to minutes spent in math and reading. 
Almost all states report at least on assessment performance indi- 
cators for students in general education, and increasing num- 
bers disaggregate and report these data for students with dis- 
abilities. Many states use assessment performance data for 
accreditation purposes while others use them for technical 
assistance, diplomas, compliance with state requirements, or to 
generate local, district, and national comparisons. Many states 
use tables or spreadsheets, and some use the Internet to com- 
municate their educational results (Ysseldyke & Nelson, 1998). 
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5. Audiences for Assessment Reports 



Reporting state assessment performance can be complex 
and controversial, making the job of communicating test results 
in an understandable and clear way very difficult (Horowitz, 
1998). However, everyone from parents to policymakers now is 
demanding more accountability from schools. According to 
Henry (1996), an important ingredient of accountability is 
“widespread reporting of the performance information that can 
stimulate enlightened action both inside and outside the educa- 
tion system” (p. 88). Henry identified several audiences for 
information on school performance, including teachers, princi- 
pals, and district-level administrators; citizens; members of the 
business community; students; parents; members of school 
boards; journalists; state department of education staff mem- 
bers; college of education faculty members; legislators; real 
estate sales personnel; and local government officials. 

The credibility of school, district, and state accountability 
efforts is largely dependent on the quality and accuracy of data 
gathering, data analysis and interpretation, and performance 
reporting. Unfortunately, too often “a single report is presented 
to an audience possessing widely varied concerns and interests, 
thus treating the public as an aggregate” (Hanson et al., 1986, 
p. 80). In addition, the cost of distributing reports is an issue 
that sometimes limits audiences. 

NASBE (1998) argues that, “An accountability system will 
serve little purpose if data are not reported in terms which poli- 
cymakers, parents, educators, and the public find useful” (p. 
32). There are many opinions about what reports should be 
like. For example, Burger (1998) suggests that reports must 
contain different levels of aggregation and detail to provide 
credible evidence of student performance to different audiences. 
The reporting format should reflect the needs of the intended 
audience. Reports should contain user-friendly text and graphics 
that provide important information to stakeholders. NASBE sug- 
gests that, to be useful, “information must be reported in for- 
mats and languages appropriate to any given audience. Plain 
language - not jargon - should be used whenever possible” (p. 
32). Reports need to use consistent terms and define any terms 
that might be unfamiliar to the intended audience. Graphs and 
charts, supplemented by concise narrative text, can clearly 
explain the data and analysis. In addition, Holcomb (1999) - 
argues that benchmarks should be included so that the reader 
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can easily compare the information with a meaningful standard. 
Information should be disaggregated in different ways in differ- 
ent reports depending upon the interests and needs of the 
intended audience. 

Audiences for performance reports are frequently divided 
into two groups: (a) internal audiences, including teachers, 
school and district administrators, other school staff, school 
accreditation agencies, and school board members, and (b) 
external audiences, including policymakers, the business com- 
munity, parents, the media, and other stakeholders. 

Internal Audiences 

Internal audiences need detailed information about school 
performance so they can make informed decisions about stu- 
dents and schools. Performance reports also enable educators 
to see what they have accomplished. Some internal users need 
comprehensive reports; however, many internal users are very 
busy and have little experience using data to make decisions. 
Holcomb (1999) argues that the K-I-S-S principle (Keep It 
Simple and Succinct) is often an appropriate approach for 
reports. This approach makes the data easily accessible to edu- 
cators. At the same time, it is important to have more detailed 
reports with varying levels of disaggregation available for those 
internal users who need additional information. 

A. Teachers, Administrators, and Other School Staff 
School staff use performance reports to guide their deci- 
sion making and school improvement efforts. They need 
detailed but readily consumable reports that are available to all 
professional staff members, since educators are in the best posi- 
tions to act on the information (Henry, 1996). According to 
both Holcomb (1999) and Thomson (1993), expanded data 
gathering and reporting enable educators to: 

• Track school and student progress toward priority goals 
beyond minim um state requirements; 

• Use objective evidence to take responsibility for their own 
improvement; 

• Keep schools from wasting time and money on curricula 
and programs that do not work; 

• Keep schools focused on student achievement and results; 
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• Ensure that all subgroups of students are receiving a rigor- 
ous, academically challenging education; 

• Drive positive change efforts using solid data; 

• Develop strategies to improve the overall operation of the 
school; and 

• Collegially share information between school staff about 
practices that are working. 

B. School Boards 

School boards govern schools through the development of 
guidelines, policies, and regulations that provide direction for 
school staff. The most important function of the school board is 
to ensure that desired student learning is occurring. The school 
board sets the tone and direction for a district Board members 
need performance reports that provide information about what 
is happening within their district currently and over time. They 
also need to be able to compare their district with other similar 
districts and understand the standards they are using to make 
comparisons. According to Brickell and Paul (1999), many 
board members lack expertise in analyzing data, so perform- 
ance reports need to be presented in a format that provides eas- 
ily accessible, yet detailed information. 

External Audiences 

Performance reports give external audiences the informa- 
tion that they need to make informed decisions about schools 
and districts, but they run the risk of encouraging stakeholders 
to evaluate schools on the results of just a few high-stake meas- 
ures. Heubert and Hauser (1999) suggest that performance 
reports intended for external audiences include: 

• Test purpose; 

• How test results will be used; 

• Whether additional information will be considered for a 

particular decision; 

• Consequences of the test for individual students; 

• Whether the test has been validated and by whom; 



• Whom to contact for additional information (pp. 261-262). 

A. Parents. 

Parents want information that helps them understand what 
their children are learning and how it relates to what their chil- 
dren should know. Reports for parents should be jargon-free 
and available in a variety of formats (Heubert & Hauser, 1999). 
Unfortunately, educators often consider parents as the least 
important consumers of performance reports and may provide 
them with only limited information (Barber et al., 1992), such 
as media reports of test scores. Recendy, the Center for 
Community Change (2001) charged: 

Though much of this year’s Congressional debate on educa- 
tion policy has focused on the need for accountability, little 
attention has been paid to the rights of parents to be pre- 
sented with the information they need to evaluate what's 
going well- or poorly - in their children's schools. The 
collection and dissemination of school performance data 
to parents and the public is perhaps the single most impor- 
tant accountability measure that local school districts can 
implement. After all, parents and students are the ultimate 
consumers of our public education system. If anyone has 
the ability to hold schools accountable, it ought to be them. (p. 1). 

Barber et al. (1992) support the importance of informing 
parents: “If parents do not understand measures of student 
achievement, then they may not provide appropriate support for 
learning and motivation.. ..Parental attitudes and expectations 
influence children’s choices of academic courses and expecta- 
tions for success” (p. 16 ) . . 

B. Policymakers 

Policymakers need policy-relevant information that enables 
them to develop ^ thorough grounding in an issue. Dunn 
(1994) argues that the policymaking process can be viewed as a 
cycle with a number of phases: 1) agenda setting, 2) policy for- 
mulation, 3) policy adoption, 4) policy implementation, and 5) 
policy assessment. Information is needed that is applicable to 
one or more phases of this cycle. 

According to Gbulka and Derlin (1995), performance 
reports are used by policymakers to strengthen arguments for 
further reforms in order to drive continued improvement of 



How the test and the curriculum are aligned; and 



ERIC 



> porting on the State Assessment Performance of Students with Disabilities 



17 



The Educational Policy Reform 
Research Institute 



public education. For example, legislators might want to know 
whether the state is making progress in increasing the number 
of students whose school achievement is “proficient” (Jaeger & 
Tucker, 1998). Reports to policymakers need to: 

• Prioritize the points presented; 

• Use simple graphics to convey the data; 

• Focus on conclusions; 

• Anticipate negative reactions; and 

• Be timely and sensitive to time constraints faced by policy- 
makers (Dunn, 1994). 

C. Media 

The media often shape the public’s perception of education 
in the United States. The United States is a democratic society 
where the business community, philanthropic organizations, 
policymakers, and the general public all rely on the print and 
broadcast media to provide coverage of educational issues. 
Well-informed stakeholders are better able to engage in a 
knowledgeable debate about student achievement that will lead 
to improved schools. 

According to Henry (1996), the media are “primary con- 
duits of school performance information to audiences outside 
the education system” (p. 88). Whether the information that is 
presented by the media is in the best format is a question. Often 
journalists are untrained in data analysis and interpretation and 
need briefings so that they can ask questions and get clear and 
accurate answers (Henry). According to Horowitz (1998): 

Tbe news media tend to oversimplify the topic. Some 
reporters don’t fully understand tbe issues. On top of that, 
they have to convey what they do know in a very small 
amount of space or time, and they have to do it in a way 
that can be easily understood by the average reader or 
viewer (p. If). 



According to Thomson (1994), reports and press releases 

prepared for the media should: 

• Focus attention on instructional improvement; 

• Stimulate interest in organizational responses that will 
enrich learning opportunities for all students; 

• Provide clear explanations of key concepts and provide 
supporting empirical evidence; 

• Put the most important information at the beginning of the 
report; and 

• Avoid the use of educational jargon. 

D. Other Audiences 

There are several additional audiences for reports on edu- 
cational performance. These audiences need access to forums 
to discuss the use of information for improvement, to register 
their concerns and to contribute their support to schools 
(Henry, 1996). Business audiences, concerned about the cost of 
education, want impartial evidence of school performance. They 
also want to see evidence that public schools are educating stu- 
dents who will be prepared to work and live in a rapidly chang- 
ing, technologically advanced society (Minnesota Planning, 
2000). Members of the business community routinely use data 
to make decisions, so they want reports that contain objective 
cost-benefit analyses. The reports should enable them to spot 
trends and identify best practices (Wall Street Journal, 2001). 

Realtors represent another important audience. According 
to Gbulka and Derlin (1995), “Realtors provide a continuing, 
although usually unintended, additional means of distributing 
public information about the schools. The impact of this 
expanded access to information about school performance con- 
tributes to continuing pressure to improve the public schools in 
local communities” (p.13). 
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6. Current Reporting Practices 



NCEO has conducted four analyses to determine what types 
of information are provided on students with disabilities in state 
education reports to the public (Bielinski et al., 2001; Thurlow, 
Langenfeld, Nelson, Shin, & Coleman, 1998; Thurlow, Nelson, 
Teelucksingh, & Ysseldyke, 2000; Ysseldyke, Thurlow, 
Langenfeld, Nelson, Teelucksingh, &Seyfarth, 1998). Only the 
last two analyses of state reports include results following the 
IDEA requirements for public reporting. 

In its first analysis of state reports, which included those 
collected between the fall of 1995 and spring of 1997, only 1 1 
states had included test-based performance data on students 
with disabilities (Thurlow et al., 1998). In contrast, 30 states 
had included data on students with disabilities that reflected 
process indicators (e.g., enrollment, attendance). 

In a second analysis, which collected data through the sum- 
mer of 1998, 13 states included test performance results for 
students with disabilities, an increase of just two states 
(Ysseldyke et aL, 1998). It was in this report that a discrepancy 
between the number of states reporting performance data and 
the number reporting participation data was first noted - while 
13 states reported on test performance, only 12 states reported 
on participation of students with disabilities in assessments. 

The third NCEO analysis was conducted in 1999, with the 
collection of reports produced between March 1998 and March 
1999 (Thurlow et al., 2000). This analysis found a total of 17 
states publicly reporting test performance results for students 
with disabilities, just up 4 from the previous year. A total of 14 
states, up 2 from the previous year, reported data on the partici- 
pation of students with disabilities in assessments. The third 
analysis of state reports also examined the other types of out- 
come data available in public reports that were disaggregated 
for students with disabilities: dropout data (8 states), gradua- 
tion/exit data (9 states). Still, the number of states reporting any 
type of outcome data was not close to the number of states 
reporting on the enrollment of students with disabilities (38 
states). Further, Thurlow et al. (2000) found that more than 
two-thirds of the documents that did not include data on stu- 
dents with disabilities did include data on the performance of 
other students. 



NCEO is currendy completing its fourth analysis of state 
reports, which is examining reports of data for the 1999-2000 
school year (Bielinski et al., 2001). This analysis found that 37 
states publicly reported performance data for some or all of 
their tests; only 20 of these reported on all of their tests. A total 
of 33 states reported data on participation in assessments for 
some or all of their tests; only 14 of these reported participation 
data for all of their tests. However, the ratio of tests for which 
disaggregated data were reported seemed to be increasing. Of 
the 66 tests reflected in the state reports, disaggregated per- 
formance data were reported for 65 of them, and disaggregated 
participation data were reported for 54 of them. 

NCEO’s fourth analysis also examined the extent to which 
states included data from alternate assessments. Of course, the 
implementation of and reporting on these assessments was not 
required until July 2000, so their analysis was ahead of the 
requirement Indeed, only one state had any information on its 
alternate assessment (Kentucky), and this state reported only 
participation data. 

Thus, as the most recent analyses indicate, states are still 
moving toward the IDEA directive to report on the participation 
and performance of students with disabilities with the same fre- 
quency and in the same detail as for other students. While there 
have definitely been increases in the reporting of performance 
data for students with disabilities, it has not been to the extent 
that might be expected given the timelin es in IDEA 97. 

Reporting Practices in Four States 

The differences among states in their reporting practices 
and assessment processes are clearly illustrated through the 
example of four states: Texas, New York, Maryland, and 
California. The practices are described here, both in general 
and as they relate to students with disabilities. These states have 
different assessments, different content and performance stan- 
dards, test at different grade levels, and use results for different 
purposes. 
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A. Texas 

Texas issues an Academic Excellence Indicator System 
(AEIS) Report to each school district. The report includes these 
student performance indicators: 

• Percentage passing the TAAS, a criterion-referenced assess- 
ment designed to measure competency in the Texas 
Essential elements curriculum; 

• Progress of prior year TAAS failers; 

• TAAS participation, performance on end-of-course exams; 

• Attendance rate; 

• Dropout rate; 

• Percent completing advanced courses; 

• Completion rate/student status; 

• Percentage of graduates completing the recommended high 
school program; 

• AP/IB (Advanced Placement/International Baccalaureate) 
results; and 

• College admissions tests results. 

Passing rates for the four end-of-course examinatio n s will 
be reported beginning in 2003, with the first test administration 
disaggregated by student group. 

TAAS results from students receiving special education serv- 
ices are included in the Academic Excellence Indicator System 
that is used to create a portrait of academic progress. Students 
can take TAAS or a state developed alternative assessment, based 
on the level of the curriculum at which the student is receiving 
the majority of his or her instruction and the use of accommo- 
dations and/or modifications. 

In the 2000 review of Texas’ assessment system under Title 
I requirements, Texas was working on a plan to increase the 
number of students with disabilities included in the TAAS assess- 
ments, since in 1999-2000 approximately 50% of students with 
disabilities were exempt from TAAS for grades 3-8 and 10. The 



state also explained how the results from alternative and alter- 
nate assessments will be reported and included in the stan- 
dards-based measures for school accountability, and confirmed 
that the results for all students, including those students taking 
the alternate assessments, will be publicly reported and includ- 
ed in measures of school progress. 

B. New York 

Previously, students who successfully passed Regents exams 
in eight subjects received a Regents (State) diploma. For many 
years, this assessment was thought to be appropriate only for “ 
college-bound students. Non-college bound students would take 
a local exam, and receive a local diploma. Thus there was diplo- 
ma choice. Beginning with the class of 2000, new “high-stakes” 
Regents exam requirements are being phased in. All students 
who graduated in 2000 took a Regents English exam and had to 
pass it at the 55% level to receive a diploma. The class of 2001 
also has to pass exams in algebra and geometry, the class of 
2002 must pass additional exams in global studies and 
American history, and in 2003, laboratory science exams will be 
required, for a total of five exams. All students will be required 
to pass these exams in order to receive a high school diploma 
A local diploma will not be available to any student. 

Data are used for student, school and district accountability 
systems (district accountability only under Tide I). Schools and 
districts also use data in improvement planning: Policies con- 
cerning rewards for high achieving schools or systems are cur- 
rendy under development. 

The performance of all continuously enrolled students, 
both general education students and students with disabilities, is 
included in the calculations used to determine whether a district 
is required to develop a Local Assistance Plan for a school; to 
identify which schools are farthest from state standards; and, for 
schools receiving Tide I funding, to determine which schools 
are making adequate yearly progress. On the school report 
card, the performance of all students tested (including students 
with disabilities) will be aggregated. On the 5th grade writing 
test, the performance of general education students will be 
reported separately from that of students with disabilities receiv- 
ing supplemental services. For all measures, except attendance, 
dropout, and suspension rates, the performance of students with 
disabilities will be displayed separately as well as agg r eg a ted . 
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For each assessment, the school is to report the number of stu- 
dents tested, the number of students scoring at each perform- 
ance level, and the number of students who were exempted by 
their IEP teams. This is a period of transition in the state, and 
the implementation of these new assessments will be at issue for 
the next few years. Also at issue are the accommodations and 
modifications for students with special needs and the challenge 
of addressing those needs across the state while providing valid 
and reliable assessments. 

In the 2000 review of New York’s assessment system under 
Title I requirements, New York is modifying existing school 
report materials to include all disaggregated data and perform- 
ance comparisons as required by Title I and providing assur- 
ance of local distribution of individual student reports. 

C. Maryland 

The Maryland State Performance Assessment Program 
(MSPAP) has long been the established testing system in the 
state. Students in grades 3, 5, and 8 are tested in reading, math, 
writing, language usage, science, and social studies. The 
Maryland School Performance Report is released in two parts: 
state and school districts (published by the state education 
department) and school systems and schools (published by 
local education agencies). Data include MSPAP results, atten- 
dance, dropout rate, enrollment, students receiving special serv- 
ices, student mobility, high school program completion, other 
factors, and gifted and talented student information. 

State policy is to include all students to the fullest extent 
possible. Maryland was fully approved in the 2000 Title I review 
of their assessment system, under the assumption that they 
would not only report the results of their alternate assessment, 
but also use those results for accountability purposes. 



D. California 

The Academic Performance Index (API) is the cornerstone 
of the 1999 Public Schools Accountability Act (PSAA) in 
California. It measures the academic performance and progress 
of schools. School districts are required to administer the 
Stanford 9 to all students in grades 2 through 11, except for 
those students whose IEPs explicitly exempt them, or students 
whose parent or guardian submits a written request to exempt 
the student. Beginning in 2004, students in California public 
schools will have to pass the High School Exit Exam to receive a 
high school diploma. 

The IEP team can exempt “individuals with special needs” 
from all or part of California’s assessment program. Severely 
disabled students whose instructional program includes func- 
tional, non-academic content and for whom there are no appro- 
priate testing accommodations can be exempt and participate in 
an alternate assessment. Student scores are excluded if: (a) the 
pupil first attended the district in the current year as indicated 
on the STAR answer document, (b) the test administration 
accommodation for the pupil is more than one grade out of 
level, or (c) any of the following four test administration accom- 
modations are marked “yes" for all content areas: Braille, flexi- 
ble scheduling, revised test format, or use of aides. 

The 2001 Title I review of California’s assessment system 
found that California needed to align its state assessments with 
its content and performance standards, and to reduce the high 
rate of exclusion of students from school accountability. 
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7. Issues in Reporting State Assessment 
Performance of Students with Disabilities 



Several issues arise in reporting the state assessment per- 
formance of students with disabilities. First, for several reasons, 
assessment results of many students with disabilities are still not 
reported. Second, alternate assessments are so new that many 
states have not yet figured out how to aggregate results with 
those of general assessments. Finally, caution needs to be taken 
when looking at the performance results of students with dis- 
abilities over time to make sure that the same group of students 
is being compared each year. 

Whose Data Still Are Not Reported? 

In a survey of stale directors of special education, NCEO 
found that almost all states reported the performance of stu- 
dents using approved accommodations, but just over half 
reported the scores of students who used non-approved accom- 
modations (Thompson &Thurlow, 2001). About the same num- 
ber of states reported scores of alternate assessment partici- 
pants; however, about one third of states had not yet made a 
decision about how to report these scores. Of the 17 states that 
used out-of-level tests at the time of the survey, 13 reported the 
scores of students who took tests designed for a lower grade 
level. Some states gave a score of “1" or “0” to students who 
were not tested (e.g., students who were absent on test days 
were counted and given the lowest possible score). 

Most states aggregate the scores of assessment participants 
using accommodations that the states view as not changing the 
test (i.e., approved accommodations) with those of all other 
assessment participants. Only half of the states that report the 
scores of students using non-approved accommodations aggre- 
gate those scores; other states report scores of these students 
separately or at the lowest score level. Of the states that have 
scoring systems in place for alternate assessments, most report 
scores separately from those of general assessment participants. 
States reporting scores of out-of-level test participants are split 
in their decisions to aggregate or report scores separately. 



Reporting Alternate Assessment Data 

A student who is not able to participate at all in general 
state assessments, even with accommodations, needs an oppor- 
tunity to show what he or she knows through an alternate 
assessment. There are several strategies that can be used to 
show progress toward state or local content standards through 
alternate assessments. Each state has selected its own approach 
(Thompson & Thurlow, 2001). Most states compile data at mul- 
tiple points over an extended period of time — usually most of a 
school year - using a variety of assessment strategies 
(Thompson, Quenemoen, Thurlow, and Ysseldyke, 2001). Over 
half of the states organize the data collected for a student’s alter- 
nate assessment into some type of portfolio, while others sum- 
marize the results on a checklist or rating scale. 

It is important for the general public, news media, and 
other educators to receive good information about what alter- 
nate assessment scores mean. However, there are a number of 
challenges in reporting the scores of alternate assessment par- 
ticipants, including concerns about statistical soundness and the 
purposes and focuses states have chosen for their alternate 
assessments. States are currently deciding how their alternate 
assessments will be scored and reported. Bechard (2001) iden- 
tified six models for reporting the results of alternate assess- 
ments that states are using or considering: 

Model 1: Same proficiency levels for general assessment and 
alternate assessment. 

Model 2: Different proficiency levels for general and alternate 
assessments treated as the same. 

Model 3: Different proficiency levels for general assessment 
and alternate assessment. 

Model 4: Overlapped proficiency levels for general assess- 
ment and alternate assessment. 

Model 5: Lowest possible proficiency level for alternate 
assessment. 



Model 6: No alternate assessment proficiency levels. 
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According to Bechard (2001), “The variety of methods cre- 
ated to report the results of alternate assessments demonstrate 
the struggle of states to incorporate these new assessments into 
an existing structure - one that previously did not have to 
address the achievement of students with significant needs, or in 
many cases, even their presence” (p. 14). 

Reporting on One Year’s Performance Versus Trends in 
Performance 

It is difficult to find large-scale performance trend data for 
students with disabilities, and the few results that have been 
reported show an achievement gap between students with dis- 
abilities and their non-disabled peers that grows steadily across 
grades (Thurlow et al., 2000). In Kentucky, Trimble (1998) 
found that the gap between students with and without disabilities 
on the 1996 reading test was less than .25 in 4th grade, almost 
.70 in 8th grade, and over .80 in 11th grade. The question con- 
sidered by Bielinski and Ysseldyke (2000) in their study of per- 
formance trends was, “Can disaggregated test results for stu- 
dents with disabilities be used to make valid inferences about 
whether students with disabilities are benefiting from education 
to the same degree as their non-disabled peers?” 

Using a large longitudinal database, Bi elins ki and Ysseldyke 
(2000) examined assessment performance trends for students 
receiving special education services. There were two particularly 
important results in this study. Erst, in each year, the average 
test score for the students leaving special education was much 
higher than it was for those who remained in special education. 
The other important result was that the students leaving special 
education were replaced by a much lower achieving group of 
regular education students. They found that special education 
services were terminated for the highest performers and initiat- 
ed for low performers who had been referred to and found eli- 
gible for services. The result was a substantial increase in the 



performance gap over time between students receiving and 
not receiving special education services. They also found that 
the reduction in exemption rates from testing that has occurred 
over time added to the size of the gap. Results suggested that the 
students from the special education group who were exempt 
from testing in the earlier grades were lower achievers com- 
pared to those who were tested every year. When the same 
group of students (who received special education services at 
least initially) was tracked over time, the performance gap actu- 
ally decreased slightly. 

These findings have significant implications for states as 
they begin to publicly report disaggregated data on students with 
disabilities, particularly if attempts are made to track perform- 
ance across time. Failure to document and account for changes 
in students’ special education status and previous exemption 
rates from testing could result in misinterpretations about the 
effectiveness of special education services. By restricting the 
group of students for longitudinal analysis to those who received 
special education services during the first year of analysis (and 
following their performance regardless of whether they contin- 
ued to receive special education services), a more accurate 
indication of progress over time will be obtained. Thus, states 
should consider ways to both report on the performance of all 
students with disabilities and the longitudinal performance of 
clearly defined targeted groups of these students. 
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8. Frameworks for Evaluating 
Reporting Practices 



In 1997 the National Center on Educational Outcomes 
(NCEO) developed a set of principles for states to consider in 
the development of policies and practices for reporting assess- 
ment data for students with disabilities (Erickson et al., 1997). 
NCEO is currendy working on an updated set of principles and 
characteristics of inclusive assessment and accountability sys- 
tems (Thurlow, Quenemoen, Thompson, & Lehr, 2001). At this 
time, six core principles of assessment and accountability sys- 
tems have been identified that include all students, specifically 
students with disabilities. Both the early set of principles and 
those recendy developed are described here. We also include 
guidelines for desirable characteristics in reporting that state 
department personnel could use to improve their accountability 
reports. These were developed in 1998 by NCEO through a study 
to identify stakeholders’ views of desirable characteristics of 
reports (Nelson, Ysseldyke, & Thurlow, 1998). 

Early Principles to Guide Reporting Practices 

These principles were based on an NCEO analysis of state 
practices in 1997. 

Include data from all test takers in performance 
reports. Even with the active participation of students with dis- 
abilities in assessment programs, exclusion can still occur at the 
stages of data aggregation and analysis. In the past, some states 
were found to remove the scores of students with disabilities 
from their testing database before further analyses were con- 
ducted. Others removed the scores of those students who took 
the tests under accommodated conditions, even if the accommo- 
dations had no impact on the test’s validity. 

NCEO recommended that accountability reports include 
testing information on all students who took the test, either with 
or without accommodations. If particular accommodations have 
the potential of invalidating test scores, such scores could be 
reported separately until research is conducted to support or 
discourage this separation. 

How performance on alternate assessments should be 
reported was still under debate in the late 1990s. Some states 
devised ways to merge data from alternate assessments with data 
from regular assessments by using a common scoring rubric 
(Bechard, 2001). Other states planned to report data from their 
alternate assessments separately from regular assessment data. 



Include rates of exclusion that are specific to stu- 
dents with disabilities and the reasons for the exclusion. 
In states with assessments that have high stakes for teachers, 
schools, or administrators, students with disabilities were 
encouraged not to participate in testing or to participate in the 
alternate assessment, due to fears that their scores would lower 
overall school or district averages. NCEO recommended that 
these practices be curtailed by requiring that exclusion rates be 
included in any public reporting of test results. 

Reporting rates of exclusion, however, should not be limit- 
ed to those situations in which there are high stakes. It is always 
important to report on exclusion so that comparability of results 
can be assessed. Federal law now supports this by requiring that 
the number of students with disabilities taking the statewide 
assessment be reported along with performance results. 

Although not required by IDEA, reporting the reason for 
exclusion can assist in pinpointing ways to increase participa- 
tion of students with disabilities in assessments. For example, 
high rates of exclusion due to absenteeism suggest a different 
issue from high rates of exclusion due to IEP team decisions 
that students are unable to participate due to emotional distress. 
Making this information public through reporting can drive 
changes in inappropriate practices. 

Calculate participation or exclusion rates using con- 
sistent written guidelines. In order to better ascertain how 
students with disabilities are being included in statewide assess- 
ment programs, states need to provide schools and districts bet- 
ter direction about how participation rates should be deter- 
mined. In the past, students with disabilities at the age or grade 
level being tested often were considered ineligible for testing, 
and were systematically excluded from testing populations 
because of their program setting or type of disability. This form 
of exclusion led to inflating the reported rates of test participa- 
tion because those excluded were not even considered eligible. 
While this type of exclusion should be a thing of the past, keep- 
ing track of any exclusions that occur is an important step in 
knowing that the intent of the law is followed. 
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Maintain records in such a way that data for stu- 
dents with disabilities can be reported separately, over- 
all, or by other breakdowns. To investigate whether new 
programs, methods of instruction, or curricula are affecting stu- 
dent performance, schools and districts may find it useful to 
break out performance data not just between special and regu- 
lar education, but also by different service delivery models, dis- 
ability categories, or types of accommodations requested. The 
question of aggregating or disaggregating test scores for stu- 
dents with disabilities rests on the purposes underlying the 
assessment. If used for holding schools and districts account- 
able for the success of all students, then aggregating the scores 
from all students is an equitable approach to reporting. 

However, if test results are intended to assist in making pro- 
grammatic improvements and curricular evaluation, then disag- 
gregation became equally important. 

Keep records of the use of accommodations accord- 
ing to the type of accommodation. Testing accommoda- 
tions are needed to allow some students with disabilities to par- 
ticipate fully in assessment programs. An accommodation can 
take different forms, from providing extended time to a student, 
to offering a scribe to record the student’s responses. 
Documenting the use of accommodations offers several benefits: 
(a) It gives testing administrators the opportunity to enforce 
policies that limit accommodations to those that the student uses 
during instruction, and (b) it provides information for further 
research about their use in testing. 

Inform parents about the reporting policy for their 
child’s data- Parents of students with disabilities need to be 
aware of how their children’s scores will be used in the public 
reporting of results. Parents should be given notification as to 
whether test scores for their students will be aggregated along 
with those of students who do not receive special education 
services. They also should understand any implications of using 
accommodations during testing, such as the removal of accom- 
modated test scores from overall analysis or reporting. And they 
should be shown how participation in testing programs can ulti- 
mately improve the educational opportunities provided to their 
children. 



Updated Principles to Guide Reporting Practices 

These principles, recently completed by NCEO (Thurlow et 
al.» 2001), are consistent with the requirements of current fed- 
eral laws governing special education and Title I services, but 
they go beyond the letter of the law where research and practice 
have suggested benefits of specific approaches. The purpose of 
these principles is to focus and clarify stakeholder discussion on 
essential components of inclusive systems and to provide an 
impetus for revisiting basic assumptions and beliefs about 
emerging state and district systems. One of these principles 
(Principle 3), with its supporting characteristics, provides the 
first level of accountability for the scores of students with dis- 
abilities: 

Principle 3. All students with disabilities are included when 
student scores are publicly reported, in the same frequency 
and format as all other students, whether they participate with 
or without accommodations, or in an alternate assessment. 

• Characteristic 3.1. All students in all placement settings 
who receive educational services are accounted for in the 
reporting system. Every student must be counted and count 
This includes students in traditional public school place- 
ments, but also includes the participation and performance 
of students who change schools or placements. All students 
receiving federally funded educational services in non-tradi- 
tional settings should be included and reported as well, 
such as students in home schools, private schools, charter 
schools, state-operated programs, and in the juvenile jus- 
tice system. The challenge of counting every student, and 
ensuring each student’s progress counts is fundamental to 
the success of standards-based reform. There is a national 
consensus that all students are to be held to high standards, 
and all schools are to fully support all student’s efforts to 
reach those standards, regardless of the setting. If some stu- 
dents are excluded or set aside in reporting, the public has 
no way of knowing how all students or all schools are 
doing. 
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• Characteristic 3.2. The number and percentage of stu- 
dents not in the assessment system in any way (with or 
without accommodations, or via an alternate assessment) 
are reported and an explanation given for their nonpartici- 
pation. At a minimum, every student who does not partici- 
pate actively in the assessment system must be detectable 
when scores are reported. Typically, this is done by report- 
ing the number of students not participating in the assess- 
ment system. Even if a state or district factors students who 
do not take the assessment into the reported scores (e.g., 
by giving them a score of zero), the number of students 
excluded should still be reported. In addition, the reasons 
for exclusion (e.g., parent request, absenteeism, noncom- 
pliance, invalid test protocol) should be reported by 
subpopulation. 

• Characteristic 3.3. Scores are reported for those students 
who take the assessment in a way that produces scores that 
are not being aggregated with other scores. Scores of stu- 
dents who take assessments with accommodations that are 
considered to reduce the validity of the score should still be 
reported, with an explanation of why they are separated 
from the scores of other students, if they are. Similarly, the 
performance of alternate assessment participants should be 
reported, with an explanation of why they are separated 
from the scores of other students, if they are. 

• Characteristic 3.4. Reports are provided to educators, 
parents, students, policymakers, and journalists, with a 
clear explanation of results and implications. State and dis- 
trict staff have a responsibility to ensure that data are used 
in ways that are consistent with the purpose of each assess- 
ment. Reports should be readily available and accessible, 
and should include cautions about misinterpretation of 
data. Data should be suppressed when low numbers may 
compromise student privacy. Consideration should be given 
to having community information sessions or special out- 
reach to the media to help people use the reports responsi- 
bly. Finally, for students in placements other than the 
“neighborhood" school, students should be included in 
reports that will most directly affect the student’s education 
- where his or her performance counts, and where public 
reporting can make a difference. For example, if a student 







with disabilities is being served in a specialized setting out- 
side of his or her home district (or school), the progress of 
that student should be reported in the context where 
responsibility and concern for that student most directly lies. 

Criteria for Good Reports 

In 1998, NCEO conducted a study to identify stakeholders’ 
views of desirable characteristics of reports, and to provide 
guidelines for desirable characteristics in reporting that state 
department personnel could use to improve their accountability 
reports (Nelson et al., 1998). A descriptive study showed that 
none of the states’ accountability reports that were examined 
met all of the desired characteristics. In a related report, 
Ysseldyke and Nelson (1998) identified necessary, desirable, 
and succinct characteristics of good state and district education- 
al accountability reports: 

• Be clear about who the report is directed to, the intended 
purposes of the report, and the state’s conceptual model for 
its accountability system. 

• Be comprehensive yet concise in the reporting of inputs, 
processes, and results for students, especially students with 
disabilities. 

• Provide comparative information with changes over time 
between schools, districts, states, regions, or standards. 

• Be concise and use carefully chosen indicators so that no 
more information is given than is necessary. A multi-layered 
approach may be appropriate. 

• Include cautions against misinterpretations of the data or 
against any unintended consequences. 

• Maintain confidentiality and avoid the possibility of identi- 
fying individual students. 

• Use good formats so that reports are well organized, read- 
able, and interesting. 

• Use catchy titles , pictures, or other visual aids to capture 
and hold the audience’s attention. 
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This paper has presented several important issues relating 
to reporting the performance of all students, including students 
with disabilities. As shown in the four states described earlier, 
assessments and reports vary tremendously from state to state. 
States need to report the academic performance of students with 
disabilities with the same regularity as they do for students with- 
out disabilities. Such reporting has benefits beyond meeting fed- 
eral mandates. Including students with disabilities in education- 
al accountability systems and reporting assessment results helps 
ensure that they do not fall through the cracks of educational 
systems (Erickson et al., 1997). 

There are several criteria that can be used to guide both 
reporting practices and the production of good reports. As seen 
in this report, these criteria have been refined recently, moving 
from basic processes to actual principles and characteristics to 
guide fundamental reporting decisions. 

Reporting assessment performance data is a critical step in 
improving educational services for all students. It is not, howev- 
er, the final step. Performance information must be used to 
improve student outcomes, and students with disabilities must 
be included in data-driven efforts to make improvements in the 
achievement levels of all students. Law and practice in most 
states require that in every district or school, parents, teachers, 
administrators, and community partners sit down together, look 
at the achievement data of all their children, and determine how 
to help all children succeed (Thompson et al., 2001). This typi- 
cally is called the “school improvement planning process,” and 
most states provide a framework to help districts and schools 
use data based decision making to make year-by-year progress 
in increasing achievement levels. In order to be effective, these 
processes must include every child, and must focus on raising 
the achievement levels of all children. 
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